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RECOGNITION -PRIMED  DECISION  STRATEGIES:  FIRST -YEAR  INTERIM  REPORT 


EXECUTIVE  SUMMARY 


The  goal  of  year  1  vas  to  expand  our  understanding  of  recognitlonal 
decision  making,  specifically  in  terms  of  hov  situation  assessments  are 
communicated.  Three  studies  were  proposed,  and  ve  have  completed  tvo  of  them 
and  have  initiated  the  third.  In  addition,  ve  have  made  an  earlier  start  of  a 
project  originally  planned  for  year  2. 

The  projects  completed  vere  a  literature  review  in  the  area  of  situation 
assessment  and  a  multidimensional  scaling  experiment  of  expert  and  novice 
firefighters.  This  experiment  vas  one  of  the  first  to  use  situations  as 
stimuli,  rather  than  simple  cues  such  as  words.  The  results  suggested  that 
experienced  fireground  commanders  do  not  have  static  schemata  for  firefighting 
situations;  there  vas  low  consistency  between  ratings  made  during  different 
sessions.  In  addition,  the  importance  ratings  for  concepts  vere  influenced 
by the  goal  context. 

Tvo  studies  vere  initiated,  to  be  completed  during  year  2.  One  vas  a 
research  project  on  commander’s  intent.  Ve  have  hypothesized  that  intent 
statements  in  terms  of  actions  to  be  performed  will  be  easier  to  follow  but 
will  not  be  adaptive  to  changing  dynamics,  whereas  intent  statements  in  terms 
of  objectives  will  allow  more  effective  improvisation  to  meet  changes  in  the 
situation.  The  subjects  for  this  study  will  be  graduate  students  in  an  M.B.A. 
program.  The  second  study  vas  an  examination  of  errors.  Ve  have  had  the 
opportunity  to  study  errors  in  team  decision  making  in  a  number  of  contexts, 
and  ve  are  using  these  data  to  develop  tvo  models  of  team  decision  making,  one 
addressing  team  decision  errors  and  the  other  focusing  on  team  decision  dynam¬ 
ics  in  operational  settings. 


Ve  also  performed  an  entirely  new  task  during  this  year- -a  workshop  on 
naturalistic  decision  making.  The  workshop  vas  held  in  Dayton,  Ohio,  Septem¬ 
ber  25-27,  1989.  Approximately  25  professionals  working  in  the  area  of 
naturalistic  decision  making  met  to  discuss  new  models  and  paradigms  and  to 
identify  directions  for  future  research.  The  majority  of  participants  rated 
it  as  more  productive  than  typical  workshops,  and  it  is  anticipated  that  a 
book  written  by  workshop  participants  will  be  completed  by  June,  1990.  The 
book  will  be  the  first  document  describing  the  recent  research  and  model 
building  in  operational  decision  making. 


A  number  of  papers  have  been  written,  submitted,  and  presented.  In  year  For 
2  we  plan  to  complete  the  tvo  projects  described  above  and  to  initiate  a  re-  H 
search  project  on  mental  simulation  of  options. 
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RECOGNITION- PRIMED  DECISION  STRATEGIES:  FIRST- YEAR  INTERIM  REPORT 


INTRODUCTION 

The  importance  of  this  basic  research  contract  is  twofold — to  extend  our 
research  on  recognitional  decision  making  into  new  areas,  and  to  identify  the 
applicability  of  the  Recognition-Primed  Decision  (RPD)  model  for  Army 
requirements  such  as  training. 

In  its  current  form,  the  RPD  model  describes  a  general  strategy  for 
making  decisions  in  complex  settings.  (See  Klein,  1989,  for  a  recent  review 
of  this  work.)  For  the  past  several  years,  supported  by  an  earlier  basic 
research  contract  from  the  Army  Research  Institute,  MDA903-85-C-0327,  we  have 
been  studying  command-and-control  decision  making  and  have  generated  a 
recognitional  model  of  naturalistic  decision  making.  We  began  by  observing 
and  obtaining  protocols  from  urban  fireground  commanders  (FGCs)  about  actual 
emergency  events  (Klein,  Calderwood,  &  Clinton-Cirocco,  1986).  Some  examples 
of  the  types  of  decisions  these  commanders  had  to  make  included:  whether  to 
initiate  search  and  rescue,  whether  to  initiate  an  offensive  attack  or 
concentrate  on  defensive  precautions,  and  where  to  allocate  resources. 

The  FGCs'  accounts  of  their  decision  making  do  not  fit  into  a  decision 
tree  framework.  The  FGCs  argued  that  they  were  not  "making  choices," 
"considering  alternatives,"  or  "assessing  probabilities."  They  saw  themselves 
as  acting  and  reacting  on  the  basis  of  prior  experience;  they  were  generating, 
monitoring,  and  modifying  plans  to  meet  the  needs  of  the  situations.  We  found 
no  evidence  for  extensive  option  generation.  Rarely  were  even  two  options 
concurrently  evaluated,  so  that  opportunities  for  tradeoffs  between  the 
utilities  of  outcomes  were  largely  absent.  We  could  see  no  way  in  which  the 
concept  of  optimal  choice  might  be  applied.  Moreover,  it  appeared  that  a 
search  for  an  optimal  choice  could  stall  them  long  enough  to  lose  control  of 
the  operation  altogether.  The  FGCs  were  more  interested  in  finding  actions 
that  were  "workable,"  "timely,"  and  "cost-effective." 

Nonetheless,  the  FGCs  were  clearly  encountering  choice  points  during 
each  incident.  They  were  aware  that  alternative  courses  of  action  were 
possible,  but  insisted  that  they  rarely  deliberated  about  the  advantages  and 
disadvantages  of  the  different  options. 

Instead,  the  FGCs  relied  on  their  abilities  to  recognize  and 
appropriately  classify  a  situation.  Once  they  knew  it  was  "that"  type  of 
case,  they  usually  also  knew  the  typical  way  of  reacting  to  it.  They  would 
use  available  time  to  evaluate  an  option's  feasibility  before  implementing  it. 
Imagery  might  be  used  to  "watch"  the  option  being  implemented,  to  discover  if 
anything  important  might  go  wrong.  If  problems  were  foreseen,  then  the  option 
might  be  modified  or  rejected  altogether,  and  a  next  most  typical  reaction 
explored. 

We  have  described  these  strategies  as  a  Recognition-Primed  Decision 
(RPD)  model  (e.g.,  Klein  et  al.,  1986;  Klein,  1989;  Klein,  Calderwood,  & 
MacGregor,  1989).  For  this  task  environment,  a  recognitional  strategy  appears 
to  be  highly  efficient.  The  proficient  FGCs  we  studied  used  their  experience 
to  generate  a  workable  option  as  the  first  to  consider.  If  they  had  tried  to 
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generate  a  large  set  of  options,  and  then  systematically  evaluated  these,  it 
is  likely  that  the  fires  would  have  gotten  out  of  control  before  they  could 
make  any  decisions.  Indeed,  research  has  shown  (Howell,  1984;  Zakay  &  Wooler, 
1984;  Rouse,  1978)  that  analytic  decision  strategies  cannot  be  effectively 
accomplished  in  a  minute  or  less. 

As  we  applied  the  RPD  model  to  different  settings  we  expanded  and 
elaborated  it.  The  current  version  is  presented  in  Figure  1.  The  simplest 
case  is  one  in  which  the  situation  is  recognized  and  the  obvious  reaction  is 
implemented.  A  somewhat  more  complex  case  is  one  in  which  the  decision  maker 
performs  some  conscious  evaluation  of  the  reaction,  typically  using  imagery  to 
uncover  problems  prior  to  carrying  it  out.  The  most  complex  case  is  one  in 
which  the  evaluation  reveals  flaws  requiring  modification,  or  the  option  is 
judged  inadequate  and  rejected  in  favor  of  the  next  most  typical  reaction. 

The  model  is  characterized  by  the  following  features: 

Situational  recognition  allows  the  decision  maker  to  classify  the  task 
as  familiar  or  prototypical.  The  recognition  as  familiar  carries  with  it 
recognition  of  the  following  types  of  information:  plausible  goals,  cues  to 
monitor,  causal  dynamics  to  monitor,  expectancies  about  the  unfolding  of  the 
situation,  and  typical  reactions. 

’  Options  are  generated  serially,  with  the  most  typical  as  the  first  one 
considered.  Option  evaluation  is  also  performed  serially,  involving  testing 
the  adequacy  of  the  option,  and  trying  to  identify  weaknesses  of  that  option 
and  find  ways  to  overcome  them. 

A  key  process  for  evaluation  is  the  use  of  mental  simulations  to  portray 
the  way  the  option  would  be  implemented  in  that  specific  environment,  to  allow 
the  decision  maker  to  detect  possible  barriers  or  opportunities  in  the 
specific  environment,  and  to  help  the  decision  maker  improve  options.  Mental 
simulations  can  also  alert  the  decision  maker  to  important  dynamics  and 
thereby  modify  the  situation  assessment. 

We  do  not  propose  the  RPD  model  as  an  alternative  to  analytic 
approaches.  Rather,  we  postulate  that  recognitional  and  analytical  decision 
strategies  occupy  opposite  ends  of  a  decision  continuum  similar  to  the 
cognitive  continuum  described  by  Hammond,  Hamm,  Grassia,  and  Pearson  (1984). 

At  one  extreme  are  the  conscious,  deliberated,  highly  analytic  strategies  such 
as  Multi-Attribute  Utility  Analysis  (MAUA),  and  Decision  Analysis.  Slightly 
less  analytic  are  strategies  such  as  Elimination-by-Aspects.  At  the  alternate 
end  of  the  continuum  are  Recognition-Primed  Decisions,  which  involve  non¬ 
optimizing  and  non-compensatory  strategies,  and  require  little  conscious 
deliberation.  The  RPDs  are  marked  by  an  absence  of  comparison  between  various 
options.  They  are  generated  by  a  starting  point  that  involves  recognitional 
matches  that  evoke  generation  of  the  most  likely  action  in  the  situation. 

We  have  tested  applications  of  the  model  in  a  variety  of  tasks  and 
domains,  including  fireground  command,  battle  planning,  critical  care  nursing, 
corporate  information  management,  and  chess  tournament  play  (Klein  et  a 1., 
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1986;  Thordsen,  Galushka,  Klein,  Young,  &  Brezovic,  1987;  Crandall  & 
Calderwood,  1989;  Crandall  &  Klein,  1988;  and  Calderwood,  Klein,  &  Crandall, 
1988).  These  studies  have  shown  good  support  for  the  validity  and  utility  of 
the  model  as  it  applies  to  individual  decision  makers. 

Coals  of  the  Project 

Three  general  goals  are  guiding  this  project: 

(1)  Evaluate  the  relative  strengths  and  weaknesses  of  the  RPD  strategy. 
We  have  been  primarily  concerned  with  describing  how  decisions  are  actually 
made  under  realistic  conditions.  Up  to  now,  we  have  had  no  way  of  determining 
when  this  strategy  was  ill-advised  and  may  have  resulted  in  less  than  optimal 
outcomes.  Although  we  have  generally  been  impressed  with  the  quality  of 
decisions  made  by  experienced  personnel  under  highly  uncertain  and  risky 
conditions  (whereas  many  other  researchers  have  tended  to  emphasize  the  non- 
optimality  and  bias  of  human  decision  making),  we  need  to  put  the  RPD  model 
into  perspective  and  ask  under  what  conditions  it  is  non-optimal  and  even  what 
particular  errors  and  biases  result  from  RPD  decisions.  We  are  particularly 
interested  in  examining  the  use  of  "mental  simulation"  as  an  evaluation 
strategy.  This  is  the  case  in  which  the  decision  makers  use  their  powers  of 
imagination  to  implement  an  option  or  image  in  a  future  state  of  the  world, 
and,  this  "image"  is  then  evaluated  for  desirability  or  sufficiency  (c.f.. 

Beach  &  Mitchell,  1987). 

(2)  Learn  more  about  the  nature  of  situation  assessment  in  command-and- 
control  environments.  The  RPD  model  emphasized  the  fact  that  decisions  are 
not  made  in  a  vacuum.  Rather,  a  situation  is  always  perceived  in  terms  of 
some  prior  experience  with  similar  situations  as  a  means  of  understanding  what 
cues  are  important,  what  actions  are  reasonable,  and  what  is  most  likely  to 
occur  next.  We  would  like  to  develop  a  conceptual  approach  and  methods  for 
studying  situation  assessment  processes  that  will  prove  useful  to 
understanding  and  improving  decision  making.  There  are  many  potential  points 
of  intersection  with  situation  assessment  and  psychological  processes  as  they 
are  studied  in  other  areas.  For  example,  current  models  of  classification  and 
categorization  relate  to  situation  assessment,  but  surprisingly  little  relates 
specifically  to  how  natural,  complex  events  are  classified.  Field 
observations  and  cognitive  research  methods  will  be  developed  and  applied  to 
investigating  a)  what  dimensions  do  "experienced"  and  "novice"  commanders  use 
in  classifying  decision  events,  b)  how  do  different  report  formats  influence  a 
subordinate's  understanding  of  a  commander's  intent,  and  d)  do  some  types  of 
assessments  better  enable  modification  of  plans? 

(3)  Explore  issues  related  to  decision  quality  and  support  from  the  RPD 
perspective.  We  wish  to  explore  methods  for  improving  decision  making  in 
operational  environments  that  are  consistent  with  the  RPD  framework  and  to 
compare  these  methods  with  more  traditional  (e.g.,  multi-attribute  utility 
analysis)  approaches.  For  example,  we  wish  to  learn  whether  it  is  possible  to 
train  people  to  make  better  use  of  their  mental  simulation  abilities  and  to 
formulate  and  communicate  better  situation  assessments.  This  training  would 
impact  individual  decision  making,  but  it  might  also  have  implications  for 
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decision  making  in  groups.  As  part  of  this  effort,  we  will  be  giving 
attention  to  formulating  means  of  measuring  decision  quality  that  make  sense 
in  command-and-control  tasks.  For  example,  there  is  seldom  a  basis  for 
determining  an  opt imal  solution,  and  therefore  measurements  of  timeliness, 
modifiability,  and  progressive  deepening  of  options  need  to  be  considered. 

Within  these  guidelines,  there  are  a  number  of  important  and  more 
specific  questions  about  the  processes  underlying  recognitional  decision 
making,  and  these  form  much  of  the  basis  for  the  current  3-year  research 
effort. 

There  are  questions  about  the  initial  stages  of  situation  assessment, 
and  how  it  is  formed,  and  how  it  is  communicated.  There  are  questions  about 
the  process  of  mental  simulation,  and  how  it  is  carried  out.  There  are 
questions  about  the  limitations  of  recognitional  decision  making,  and  about 
the  types  of  errors  that  are  generated  by  this  strategy  as  opposed  to 
concurrent  deliberation  between  options.  There  are  questions  about  the  role 
of  recognitional  decision  making  in  a  team  setting.  Finally,  there  are 
questions  about  the  guidance  offered  by  a  recognitional  model  in  terms  of 
training. 

OBJECTIVES  FOR  YEAR  1 
.# 

For  the  first  year,  we  decided  to  focus  on  the  initial  questions 
regarding  situation  assessment  and  its  communication.  There  has  been  a  great 
deal  of  work  on  what  has  been  referred  to  as  the  "choice  point,"  or  the 
"decision  event."  There  has  been  much  less  attention  to  pre-decision 
processes,  yet  much  of  our  own  work,  and  that  of  other  decision  researchers 
working  in  operational  environments,  points  to  situation  assessment  as  usually 
more  important  than  the  decision  event. 

Accordingly,  three  studies  were  planned  for  Year  1:  a  literature  review 
of  situation  assessment,  a  controlled  study  of  the  concepts  used  by  experts 
for  situation  assessment,  and  research  on  the  way  situation  assessment  is 
communicated  in  the  field. 

ACCOMPLISHMENTS  OF  YEAR  1 

The  first  study  was  completed. 

STUDY  1 :  A  REVIEW  OF  SELECTED  LITERATURE  RELEVANT  TO  THE  COMMUNICATION  OF 
SITUATION  ASSESSMENT 

Over  20  articles  were  reviewed  that  dealt  directly  with  topics  labeled  as 
"situation  assessment"  or  "situation  awareness."  A  summary  of  this  literature 
was  prepared  and  key  articles  were  selected  to  be  used  in  planning  studies  and 
addressing  the  situation  assessment  questions  that  we  have  defined.  These 
articles  were  culled  from  a  variety  of  sources,  primarily  technical  reports. 
Many  of  these  articles  related  to  pilot  situation  awareness.  Pilot  situation 
awareness  is  used  to  define  the  unknown  dimension  that  distinguishes  the 
expert  pilot.  There  have  been  attempts  to  measure  it  (e.g.  Wells,  Venturino, 
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&  Osgood,  1988),  train  it  (e.g.  Lintern  et  al.,  1987)  and  generate  profiles  of 
personality  attributes  or  cognitive  styles  of  those  who  "have  it"  so  personnel 
can  be  selected  with  those  attributes  in  Bind  (e.g.  Hartman  4  Sechrist,  1988). 
Within  the  tactical  decision-naking  literature,  the  preferred  term  is 
"situation  assessment,"  and  this  term  seems  to  fit  more  closely  with  our 
notions  of  pre-decision  processes.  Many  investigators  in  this  area  are 
adopting  notions  from  schema  and  script  theory  (Noble  &  Truelove,  1985;  Smith 
et  al.  1986)  and  some  of  these  ideas  are  being  used  in  designing  our  study 
■ethodologies.  Some  of  this  work  has  been  directly  guided  by  recognition?^ 
models  of  decision  making  and  we  are  continuing  to  track  potential  sources. 

The  second  study  was  also  completed  during  Year  1. 

STUDY  2:  DIFFERENCES  IN  THE  KNOWLEDGE  REPRESENTATIONS  OF  EXPERT  AND  NOVICE 
FI REGROUND  COMMANDERS  USING  A  CLUSTER  ANALYTIC  APPROACH 

This  study  was  pla.ined  during  Year  3  of  our  previous  ARI  contract  MDA903- 
85-C-0327  and  was  completed  during  the  current  contract.  Thirty-two 
firefighters  (16  "experts"  and  16  "novices")  from  urban  departments  in  Ohio 
participated  in  this  study.  Subjects  provided  pairwise  relatedness  ratings 
for  231  pairs  of  fireground  concepts  (set  of  24  concepts)  in  two  separate 
sessions  separated  by  approximately  two  weeks.  Prior  to  performing  the 
ratings,  a  specific  context  was  created  by  having  domain  "experts"  and 
"novices"  first  participate  in  one  of  two  simulated  fire  incidents  that 
represented  substantially  different  types  of  tactical  problems.  Rating 
context  was  held  constant  for  one  group  of  subjects  and  was  varied  for  another 
by  showing  the  same  scenario  or  a  different  one  in  the  sessions. 

A  network  scaling  approach  (Pathfinder)  was  used  to  analyze  the  resulting 
"distance"  matrices  for  all  of  the  individual  subjects  as  well  as  matrices 
based  on  group-averaged  data.  Selected  analyses  were  also  carried  out  using 
multiple  dimensional  scaling  (MDS)  analysis.  While  it  had  been  predicted  that 
the  across-session  ratings  in  the  context-consistent  group  would  show  higher 
agreement  (using  three  different  agreement  indexes)  than  those  in  the  context- 
varied  group,  this  result  was  not  obtained.  The  context  of  the  rating  task 
did  not  appear  to  affect  the  specific  ratings  given  to  the  concept  pairs, 
suggesting  that  such  ratings  are  tapping  relatively  context-independent 
knowledge.  In  contrast,  importance  ratings  of  the  concepts  carried  out  prior 
to  the  relatedness  ratings  did  show  effects  of  the  specific  scenarios. 

Overall,  the  agreement  between  subjects'  Session  1  and  Session  2  ratings  were 
unexpectedly  low.  The  implications  of  this  result  for  human  factors 
applications  were  explored. 

A  content  analysis  of  the  Pathfinder  and  MDS  solutions  provided 
suggestive  evidence  of  differences  between  personnel  with  more  and  less 
experience  in  this  domain,  although  the  differences  were  not  as  striking  as 
those  found  in  similar  studies.  We  interpreted  the  differences  as  reflecting 
learning  of  functional  relationships  between  concepts.  In  the  Revised 
Research  Plan  we  proposed  to  examine  a  group  of  subjects  who  were  more  truly 
"novices"  than  the  firefighters  studied  here,  who  all  had  had  over  5  years  of 
operational  experience.  This  would  have  allowed  a  more  powerful  test  of  the 
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idea  that  concepts  are  progressively  redefined  in  terms  that  reflect  their 
tactical  implications.  However,  upon  further  consideration  it  seemed  to  us 
that  this  avenue  of  investigation  was  not  yielding  sufficient  information 
about  situation  assessment  or  its  communication  to  warrant  the  additional 
expenditure.  We  would  still  like  to  do  the  study,  but  in  prioritizing  our 
interests,  there  were  other  projects  that  came  out  higher.  Portions  of  this 
study  comprised  the  doctoral  dissertation  for  Roberta  Calderwood,  granted  by 
the  University  of  New  Mexico. 

The  third  study  was  initiated  but  will  not  be  completed  until  Year  2. 

STUDY  3:  THE  EFFECT  ON  PERFORMANCE  OF  THE  STRUCTURE  AND  CONTENT  OF 
COMMANDER'S  INTENT  STATEMENTS 

We  performed  field  observations  of  brigade-level  exercises  of  the  54th 
Mech  Division  at  Ft.  St  ;wart  (June  6-12)  and  National  Training  Center 
exercises  at  Ft.  Irwin,  California  (August  8-17).  While  these  were  no*- 
classical  experiments,  they  were  important  in  shaping  our  thinking. 

Initially,  we  believed  that  the  communication  of  situation  assessment  was 
often  inaccurate,  and  we  felt  that  by  observing  errors  we  would  gain  an 
understanding  of  the  process.  However,  at  Ft.  Stewart  we  found  very  few 
errors.  Instead,  the  point  of  vulnerability  appeared  to  be  the  way  that 
situation  assessment,  and  particularly  Commander's  Intent,  was  communicated. 

If  the  intent  was  communicated  in  terms  of  a  series  of  acti  ins  (as  we  had 
earlier  observed  during  corps  and  division-level  exercises  at  an  Advanced 
Warfighting  course  at  the  Command  and  General  Staff  College,  Ft.  Leavenworth, 
KS)  then  there  was  little  room  for  improvisation  as  the  battle  conditions 
changed.  Examples  are  shown  in  Table  1A.  On  the  other  hand,  if  intent  was 
communicated  in  terms  of  higher-level  goals,  as  we  saw  at  Ft.  Stewart  (see 
Table  IB),  then  there  was  a  much  higher  level  of  effective  improvisation. 

This  hypothesis  was  informally  confirmed  by  our  observations  at  Ft.  Irwin,  and 
we  have  presented  these  findings  at  the  1989  MORS  conference,  held  at  Ft. 
Leavenworth,  KS.  These  have  led  to  formulation  of  hypotheses  about  how  the 
form  and  content  of  Commander's  Intent  statements  might  influence  decision 
performance  during  battle  planning. 

Based  on  these  observations  and  a  search  of  relevant  literature,  a 
preliminary  research  design  was  formulated  to  study  the  two  types  of  intent 
statements  in  a  simulated  Army  scenario. 

The  research  design  addressed  several  questions:  What  are  the  nature  of 
performance  differences  under  action-  versus  outcome-based  Commander's  Intent 
statements?  Does  one  of  these  formats  facilitate  a  more  complete  match 
between  the  performance  of  subordinates  and  the  actual  intentions  of  the 
commander?  Of  particular  interest  are  situations  where  preplanning  is  over¬ 
taken  by  current  events  and  success  depends  upon  implementation  of  an 
improvised  plan  of  action. 
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Table  1A 


COMMANDER'S  INTENT 

Ft-  Leavenworth,  Command  and  General  Staff  College,  P318,  Combat  Operations, 

Lesson  7 

Division  to  Brigade 

Attack  in  zone 

through  elements  of  the  23rd  Armored  Division 

to  linkup  with  1st  Frigade,  21st  Infantry  Division 

and  secure  objective  snake 

to  assist  passage  of  the  23rd  Division 

and  then  follow  and  support  the  23rd 

to  facilitate  the  corps  exploitation 


Table  IB 


COMMANDER'S  INTENT 

Ft.  Stewart,  1-64,  Frago  3,  DIS  (ARTEP  Maneuver,  2nd  Brigade  of  24th  Infantry 
Division) 

Brigade  to  Battalion 

The  Division  must  protect  Claxton  and  the  major  surrounding  road  network 

for  reconstruction  for  future  offensive  operations 

You  must  not  allow  the  enemy  to  penetrate  Phase  Line  Martini 

Reposition  as  necessary,  do  not  get  bypassed 

Success  is  when  we  have  destroyed  all  enemy  in  sector  and  have 
consolidated  to  meet  follow  on  echelons 
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The  research  was  designed  to  test  several  hypotheses: 

HI:  Outcome-based  Commander's  Intent  statements  result  in  truer  matches 
between  the  performance  of  subordinates  and  the  actual  intentions  of  the 
commander  in  situations  where  improvisation  is  required. 

H2:  Action-  and  outcome-based  Commander’s  Intent  statements  will  result 
in  comparable  matches  between  the  performance  of  subordinates  and  the  actual 
intentions  of  the  commander  in  situations  where  improvisation  is  not  required. 

H3:  The  offensive/defensive  nature  of  the  scenarios  will  have  no  effect 
on  the  trueness  of  match  for  either  the  improvisation  or  non-improvisation 
conditions. 

H4:  Impact  of  action-  vs.  outcome-based  Commander's  Intent  statements 
will  vary  depending  on  the  experience  level  of  personnel  attempting  to 
implement  the  operations  order.  This  possibility  was  suggested  to  us  by 
Fireground  Command  trainers  at  the  National  Fire  Academy  who  have  found  they 
must  alter  the  format  and  content  of  orders  depending  on  an  FGC's  experience 
level. 


As  originally  proposed,  we  intended  to  carry  out  this  study  using  a 
coramand-and-control  task.  In  preliminary  work,  three  types  of  intent 
statements  for  use  in  this  study  were  developed:  action  descriptions, 
specific  outcome  descriptions,  and  global  definitions  of  success.  We  also 
began  development  of  scenarios  based  on  operations  orders,  maps,  and  overlays 
from  an  actual  field  exercise.  Permission  was  sought  to  use  students  at  the 
Command  and  General  Staff  College  (CGSC)  as  subjects  in  this  study.  We  had 
worked  before  with  instructors  at  CGSC  and  were  optimistic  that  permission 
would  be  received.  Plans  were  to  run  subjects  during  the  Spring  and  early 
Summer  and  complete  the  project  by  the  end  of  this  contract  period. 
Unfortunately,  there  were  practical  barriers  to  the  use  of  subjects  at  the 
CGSC,  creating  not  only  a  delay  but  an  increase  in  the  cost  of  the  research. 

Since  this  experiment  is  so  important  to  our  understanding  of  situation 
assessment,  we  decided  to  press  on  by  shifting  domains.  The  research  is 
currently  planned  to  address  executive  decision  making  in  a  corporate 
environment.  We  will  be  contrasting  expert-novice  performance,  using 
simulated  business  exercises.  The  subjects  will  be  graduate  students  at 
Wright  State  University,  enrolled  in  an  MBA  program.  The  "experts"  will  be 
students  with  considerable  business  experience  who  are  returning  for  a  degree, 
and  the  "novices"  will  be  students  entering  the  program  directly  upon 
completion  of  the  B.A.  or  B.S.  This  will  be  a  cleaner  expert/novice  contrast 
than  we  could  have  achieved  using  Army  officers  as  subjects.  The  major 
objectives  of  this  research  will  otherwise  remain  the  same. 

ADDITIONAL  WORK 

We  have  performed  several  more  projects  that  were  not  originally 
scheduled  for  Year  1. 
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STUDY  4:  AN  INVESTIGATION  OF  ERRORS  IN  TEAM  DECISION  TASKS 


The  study  of  errors  originally  planned  for  Year  2,  has  been  moved  ahead 
into  the  current  period.  The  impetus  for  this  change  was  several  projects 
carried  out  this  year  for  the  Department  of  Energy,  NASA,  and  a  separate 
contract  with  ARI  (MDA903-89-C-0077).  These  efforts  have  provided  databases 
that  contain  much  richer  information  on  decision  errors  than  we  previously  had 
available  in  our  own  data  archives.  A  description  of  this  effort  follows: 

Study  Goals 

The  purpose  of  the  study  is  to  develop  a  basis  for  classifying  errors 
that  occur  in  decision  tasks  taking  place  in  four  operational  environments 
characterized  by  their  involvement  of  teams,  high  uncertainty  of  critical 
information  (situation  assessment),  high  risk  consequences,  and  time  pressure. 
Four  databases  were  selected  from  previous  studies,  two  of  which  were  carried 
out  for  ARI,  one  for  NASA/ Ames,  and  one  for  the  Department  of  Energy's  Central 
Training  Academy.  Details  of  these  databases  are  provided  in  the 
paper  "Cognitive  Processes  of  the  Team  Hind”  (Thordsen  &  Klein,  1989).  The 
goal  of  this  study  is  to  develop  a  taxonomy  of  team  decision  errors.  This 
project  will  also  serve  as  the  Masters  thesis  for  Marvin  Thordsen. 

<  It  is  important  to  us  to  be  studying  team  performance  at  this  point. 

Much  decision  making  occurs  in  a  team  concept,  and  we  need  to  understand  how 
the  RPD  generalizes  to  team  decision  making.  Team  decision  making  was  not  an 
original  component  of  our  research  plan,  but  we  have  become  excited  at  the 
opportunity  for  progress  in  this  area- 

Thus  far,  based  on  preliminary  analyses,  we  have  developed  two  new 
models  of  team  decision  making:  A  model  of  team  decision  errors,  and  a  model 
of  team  decision  dynamics  relying  on  the  metaphor  of  a  team  mind.  These  two 
models  will  be  described  below. 

Model  1:  Team  Decision  Errors.  Here,  we  are  attempting  to  explain  how 
team  functioning  can  degrade  decision  performance.  Specifically,  we  have  been 
developing  a  model  of  team  decision  errors  linked  to  barriers  in  recognitional 
decision  making. 

The  majority  of  research  efforts  on  errors  has  concentrated  on 
individual  rather  than  team  errors.  For  example,  an  individual  reading  a 
gauge  incorrectly  is  an  example  of  an  individual  error,  while  the  three 
members  of  an  airline  crew  having  different  understandings  of  the  functional 
state  of  the  aircraft's  generator  system  is  a  team  error.  We  have  attempted 
to  develop  a  team  decision  error  model  (Klein  &  Thordsen,  1989).  Figure  1  has 
diagrammed  the  RPD  model  for  individual  decision  making.  This  model  can  be 
extrapolated  to  the  team  setting.  Thordsen  et  al.,  (1988)  found  that  26  of  27 
decisions  identified  in  a  five-hour  Army  battalion-level  planning  exercise 
followed  the  RPD  model  rather  than  any  concurrent  option  analysis  models. 

This  was  surprising  to  us,  as  we  had  anticipated  that  teams  would  show  more 
classical  patterns  of  generating  several  options  and  analyzing  these  on  fixed 
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dimensions.  We  found  that  the  RPD  model  for  individual  decision  makers  also 
fit  team  decision  making — the  teams  were  identifying  prime  options  and 
progressively  deepening  those  options  just  as  individuals  do. 

Some  team  functions  will  be  hindered  somewhat  by  the  added  burden  of 
communication  of  intent  and  situation  assessment,  the  additional  members  of 
the  team  simultaneously  provide  a  wider  skill/knowledge  base,  more  eyes,  ears 
and  hands  to  increase  their  potential  to  handle  additional  workload.  There  is 
an  additional  tradeoff  for  this  in  the  cost  for  coordination.  Thordsen  and 
Klein  identified  eight  potential  barriers  to  team  decision  within  the  team  RPD 
model  (Figure  2).  These  include:  1)  distorted  perception,  2)  difficulties 
with  situation  assessment  handoff,  3)  difficulties  in  the  formulation  or 
transmission  of  intention,  4)  directed  attention,  5)  missing  expectancies, 

6)  restricted  improvisation,  7)  synchronization,  and  8)  meta-cognition.  These 
barriers  are  outlined  below: 

1.  Distorted  Perception.  Unlike  an  individual,  a  team  cannot  have  direct 
access  to  events.  Therefore,  perceptual  cues  available  to  the 
individual,  and  the  experience  gained  in  perceptual  learning,  may  be 
lost  to  the  team;  external  events  are  usually  not  directly  experienced, 
and  the  descriptions  are  communicated  in  verbally.  Since  these 
perceptions  are  experienced  outside  the  team,  there  is  the  additional 

,i  factor  of  the  credibility  of  the  information  source.  For  example,  in  a 
battle  field  environment, the  brigade  tactical  operations  center  (TOC) 
will  receive  reports  of  enemy  activity  from  scouts  and  field  units.  The 
enemy  activity  is  seldom  observed  directly  by  the  members  of  the  TOC  but 
is  experienced  second-hand  via  radio  reports  from  the  field.  These 
reports  are  the  perception  of  the  field  units,  and  the  TOC  personnel  may 
not  perceive  the  actual  activity  is  the  same  way  it  was  perceived  by  the 
reporting  unit.  In  addition,  occasionally  the  credibility  of  the 
reporting  individual  is  questioned  and  the  TOC's  perception  of  the 
information  may  be  altered  accordingly. 

2.  Handoff  of  Situation  Assessment.  This  is  similar  to  Distorted 
Perception,  but  it  involves  difficulties  with  the  actual  articulation 
and  communication  of  the  situation  assessment  (SA).  An  individual  can 
recognize  a  situation,  but  how  does  that  recognition  get  transmitted  to 
others  in  the  team?  This  is  another  opportunity  for  distortion  or 
omission.  For  example,  a  member  of  the  National  Security  Council  (NSC) 
may  assess  a  situation  as  being  one  where  the  Soviets  are  planning  to 
invade  Central  Europe.  However,  this  situation  assessment  is  based  on 
the  individual's  recognition  of  certain  critical  cues,  etc.  that  may  not 
all  be  that  overt.  Therefore,  when  his/her  assessment  is  communicated 
to  the  other  members  of  the  NSC,  he  or  she  may  not  be  able  to 
communicate  all  of  the  key  elements  and  cues  in  a  fashion  that 
adequately  hands-off  his  SA  to  the  others. 
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1.  Distorted  Perception 


Experience  the  Situation  in  a  Changing  Context 


Reassess 

Situation 


Seek  More 
Information 


2‘  Handoff 


Is  the  Situation 
familiar? 


3.  Commanders 
Intent 


Are  Expectancies 
violated? 


4.  Attention  | 


Recognition  has 

four  aspects 

Goals 

Cues 

Expectancies 

Actions 

5.  Anticipation 


6.  Improvisation 


7.  Synchronization  |  Mental  Simulation  of  Action  (n) 


Modality 


Will  it  work? 

i  ~  : : 


Implement 


Figure  2:  Seven  Barriers  to  the  Team  Recognition-Primed  Decision  Model 


3.  Intent.  One  aspect  of  situational  recognition  is  the  recognition  of 
goals.  An  individual  decision  maker  directly  knows  what  the  goals  are 
in  a  situation.  In  a  team,  goals  must  be  communicated  and  if  the 
communication  is  poor,  the  activity  of  the  team  can  become  confused.  It 
should  also  be  noted  that  if  the  original  formulation  of  the  intent  is 
poor,  then  additional  problems  can  arise.  An  example  of  an  intent- 
transmission  problem  was  presented  earlier,  where  the  corps  commander 
asked  a  division  commander  to  move  his  troops  forward  to  block  an  enemy 
advance.  The  troops  had  already  dug  in  their  defensive  position,  and 
would  have  to  abandon  that  position  to  reach  the  proposed  line  of 
defense.  Moreover,  there  was  a  good  chance  that  the  enemy  would  get 
there  first.  Nevertheless,  the  division  commander  issued  orders  to 
carry  out  the  intent  without  questioning  whether  the  corps  commander 
understood  the  implications. 

4.  Attention.  An  individual  naturally  looks  at  data  sources  that  are  of 
specific  interest  to  what  they  want  to  accomplish,  i.e.,  directing 
attention  to  what  is  important.  In  a  team,  the  data  gatherers  are  often 
different  from  the  synthesizers  and  planners  and  they  may  be  in 
different  places.  Therefore,  it  is  harder  to  get  the  information  that 
is  needed  at  the  moment  and  it  is  harder  to  shield  team  members  from 
irrelevant  information  that  can  create  overloads.  For  example,  the 
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Colonel  commanding  a  brigade  may,  in  a  planning  session,  see  the 
possibility  of  an  enemy  deception  taking  place.  But  because  he  is  not 
in  a  position  to  attend  directly  to  the  information  that  would  confirm 
or  deny  this,  he  is  somewhat  handicapped  because  he  is  required  to  take 
time  to  direct  the  field  units  on  what  to  look  for  and  then  to  wait  for 
the  reports  to  come  back.  Then,  when  the  reports  do  come,  many  reports 
will  probably  be  irrelevant,  unwanted  information  that  the  commander 
will  have  to  sort  through  to  get  to  what  he  desires. 

5.  Anticipation/expectancv.  An  experienced  decision  maker  can  use 
expectancies  to  check  situation  assessment  and  to  prepare  for  smooth 
reactions.  In  a  team,  these  expectancies  are  difficult  to  communicate 
and  may  be  missed,  so  team  members  may  operate  for  long  periods  of  time 
on  outmoded  and  obsolete  expectancies.  In  ineffective  teams  there  is 
often  little  anticipation,  so  the  team  is  too  often  behind  the  power 
curve  rather  than  ahead  of  it.  For  example,  an  experienced  commander 
will  be  able  to  sense  that  there  aren't  enough  resources  to  carry  out  a 
plan,  and  will  abandon  the  attempt  very  quickly,  whereas  a  less- 
experienced  commander  might  have  to  incur  significant  losses  before 
giving  up.  Working  in  a  team  setting,  it  is  harder  to  communicate 
anticipations  and  mismatches  so  that  lower  echelons  can  let  a  commander 
know  when  expectations  are  so  far  off  that  it  is  time  to  reconsider  the 

.1  mission. 

Expectancy  errors  also  include  inadequate  time  sensitivity — not 
appreciating  how  long  messages  and  orders  take  to  be  implemented,  or  how 
long  incoming  messages  take  to  be  received.  Moving  from  small-  to 
large-scale  organizations  can  be  like  the  transition  from  steering  a 
powerboat  to  steering  a  super-tanker.  If  it  suddenly  takes  five  miles 
to  make  a  turn,  the  expectancies  have  to  be  operating  much  earlier  than 
usual . 

6.  Improvisation.  Generals  know  that  even  the  best  plans  can  become 
obsolete  within  a  day  or  so.  Individuals  can  quickly  recognize  that 
assumptions  have  failed,  and  can  switch  their  assessment  to  match  field 
conditions.  Teams  have  difficulty  improvising.  They  have  trouble 
giving  up  plans  that  no  longer  make  sense,  and  they  have  trouble  seizing 
opportunities  that  were  not  expected.  For  instance,  a  battalion  may  be 
ordered  to  make  sure  that  the  enemy  does  not  get  across  a  bridge. 
However,  upon  arrival  they  may  find  the  bridge  undefended.  Should  they 
still  set  up  their  blocking  position,  or  should  they  seize  the  bridge. 
Perhaps  they  should  move  forward.  It  is  up  to  the  commander  to  provide 
a  sufficient  overview  of  intent  so  that  lower  echelons  can  initiate 
actions  effectively. 

7.  Synchronization.  Skilled  decision  makers  can  play  out  a  planned  action 
in  their  heads  to  see  if  it  will  work.  Teams  try  to  do  the  same  thing 
but  they  have  trouble  representing  all  the  different  dimensions  (e.g., 
logistics,  artillery,  intelligence,  weather,  air  support,  etc.). 

Complex  plans  can  be  stymied  by  the  mis-timing  of  only  a  few,  sometimes 
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only  one,  of  the  components  that  must  be  synchronized.  Synchronization 
implies  that  multiple  elements  and  players  must  be  sequenced  in  a  timely 
fashion  for  all  to  proceed  properly.  In  a  team  setting  these  different 
elements  and/or  phases  are  often  the  responsibilities  of  different  team 
members.  Therefore,  the  need  for  communication  of  critical  aspects  and 
needs  of  these  different  concerns  is  important  for  proper 
synchronization.  For  example,  if  the  fire  support  officer  does  not 
coordinate  and  synchronize  with  the  air  officer  in  a  miJ;tary  maneuver, 
aircraft  could  be  shot  down  by  friendly  artillery. 

8.  Meta-cognition.  Just  as  an  individual  decision  maker  must  learn  how  to 
allocate  the  proper  amount  of  resources  to  a  task,  so  must  a  team.  And 
it  is  both  more  important  and  more  difficult  for  a  team  to  do  this  well. 
Both  the  leader  and  the  team  members  must  be  able  to  communicate  about 
workload,  actual  and  impending,  so  that  functions  are  assigned 
appropriately,  and  the  allocations  are  made  in  enough  time  to  smooth  the 
performance  rather  than  disrupt  it.  The  function  of  meta-cognition  does 
not  appear  in  Figure  2  because  it  is  not  a  stage  of  decision  making,  but 
rather  the  management  of  the  decision  process. 

Model  2:  The  Team  Mind.  In  performing  studies  of  team  decision  making, 
we  (Klein  A  Thordsen,  1989;  Thordsen  &  Klein,  1989)  proposed  that  in  a 
group/team  setting  there  is  an  additional  "individual"  functioning  that  is 
best  described  as  the  result  (or  embodiment)  of  the  "team  mind."  This  is  a 
new,  preliminary  model  of  team  decision  making  dynamics  that  grew  out  of  the 
conceptual  work  we  had  performed  during  the  initial  basic  research  contract, 
and  came  into  clear  conceptualization  during  the  study  of  flight  crews  that  we 
conducted  for  NASA/Ames. 

The  model  asserts  that  the  team  mind  has  three  aspects  that  can  be 
studied:  behavior,  collective  consciousness,  and  subconsciousness.  These 
aspects  will  be  described,  along  with  methods  for  studying  each. 

1 •  The  mind  of  a  team  can  be  treated  as  analogous  to  the  mind  of  a  person. 
The  goal  of  this  analogy  is  to  help  us  understand,  study,  and  represent  the 
performance  of  a  team.  The  unit  of  analysis  is  the  team,  a  group  of 
individuals  with  common  goals  and  coordinated  roles.  A  team  may  have 
extensive  experience  working  together,  or,  like  the  crew  of  a  Boeing  727,  the 
members  may  be  meeting  each  other  for  the  first  time  just  prior  to  takeoff. 

In  contrast,  a  group  of  individuals  may  not  have  consensus  on  goals  and  roles 
so  that  interesting  features  of  teamwork  do  not  emerge.  In  this  paper  we  are 
restricting  our  speculations  to  teams,  and  we  are  postulating  that  a  team  be 
treated  as  an  individual  entity  capable  of  taking  actions  that  depend  on 
intentional  and  accidental  "mental"  processes.  The  knowledge  or  ignorance  of 
a  single  new  member  is  relevant  if  it  affects  the  actions  of  the  team. 

2.  Team  mind  has  three  aspects:  behavior,  collective  consciousness,  and 
subconsciousness  (see  Figure  3). 
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3.  The  team  is  embodied  in  the  actions  taken.  The  tea®  acts,  as  does  a 
person.  Using  the  Boeing  727  crew  as  an  example,  the  cockpit  crew  can  be 
considered  a  single  entity.  It  is  the  team  who  adjusts  the  controls  and 
issues  communications.  Any  action  or  message  is  considered  to  come  from  the 
single  entity,  the  team.  The  embodiment  of  the  team  is  its  observable, 
recordable  behavior.  If  the  Air  Traffic  Controller  issues  a  directive  to  any 
one  of  the  three  team  members,  it  is  assumed  that  the  directive  will  be  acted 
on  by  the  whole  team.  If  a  distress  signal  is  sent  out,  it  is  considered  to 
come  from  the  whole  team  regardless  of  which  crew  member  made  the 
announcement.  For  a  command-and-control  team,  the  embodiment  is  the  set  of 
plans  it  issues. 

4.  Part  of  the  team  mind  is  conscious  and  part  is  subconscious. 

4a.  The  conscious  part  of  the  team  mind  is  whatever  is  said  out  loud 
for  all  to  hear.  So  the  collective  consciousness1  of  a  team  can  usually  be 
studied  by  listening.  In  other  words,  the  content  of  collective  consciousness 
is  directly  accessible  while  the  team  is  performing  a  task.  It  can  be  studied 
without  interfering  with  the  task. 

In  contrast,  the  individual  mind  cannot  be  directly  accessed.  If  you 
ask  a  person  to  give  a  verbal  protocol,  that  will  be  incomplete,  distorted, 
and  will  affect  the  way  the  task  is  performed. 

We  are  positing  a  spotlight  model  of  consciousness  for  the  team  mind,  in 
which  team  members  publicly  share  only  a  part  of  what  they  are  noticing.  For 
example,  in  our  NASA/Ames  study  one  malfunction  was  a  leaking  fuel  tank.  When 
this  was  detected  by  the  Flight  Engineer  it  was  immediately  brought  to  the 
attention  of  the  other  two  teammates,  the  Captain  and  the  First  Officer.  At 


’This  phrase  was  suggested  by  Judith  Orasanu  and  her  colleagues  at  the 
Army  Research  Institute. 
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that  point  it  entered  the  collective  consciousness  of  the  team  and  we  could 
record  this  consciousness  on  videotape. 

4b.  Part  of  the  team  mind  is  subconscious.  This  refers  to  knowledge 

held  by  one  person  but  not  shared  with  others.  It  is  important  to  note  that 
we  are  using  "subconscious"  to  refer  to  any  material  that  is  fully  conscious 
for  one  or  more  team  members  but  has  not  been  publicly  brought  to  the 
attention  of  the  team. 

Again,  the  phenomenon  of  subconscious  awareness  is  more  easily  studied 
at  the  team  level  than  at  the  individual  level.  Videotapes  and  interviews 
will  show  when  one  team  member  had  knowledge  not  available  to  the  others.  For 
example,  in  the  NASA/ Ames  study,  Crew  #3,  Segment  $ 3,  the  Flight  Engineer 
noticed  that  the  number  7  slat  was  not  extending  as  they  were  getting  ready  to 
land.  He  dug  out  his  checklist  before  saying  anything  to  the  other  two  team 
members,  who  quickly  detected  the  problem  themselves.  He  assumed  they  would 
find  the  problem,  and  he  was  preparing  for  their  inevitable  request  that  he  go 
through  his  checklist  of  landing  problems.  But  until  the  other  team  members 
noticed  the  problem,  it  was  subconscious  for  the  team  mind. 

Understanding  has  entered  collective  consciousness  when  all  or  most  of 
the  team  members  are  aware  of  information  and/or  intentions.  This  joint 
awareness  is  achieved  by  discussion,  or  by  watching  for  non-verbal  cues, 
including  sounds  of  switches  being  thrown  or  the  sounds  of  flap  wheels 
turning.  Cues  are  shared  and  crew  members  know  the  cues  have  been  shared  and 
observers  can  generally  detect  the  sharing  as  well. 

The  model  of  team  mind  will  be  evaluated  on  how  well  it  enables  us  to 
generate  hypotheses  about  team  decision  performance.  Our  expectation  is  that 
a  variety  of  cognitive  phenomena  will  be  applicable  to  the  team  mind,  and  the 
process  of  evaluating  these  phenomena  will  clarify  team  performance  issues. 

We  will  briefly  describe  a  number  of  hypotheses  suggested  by  the  model  of  team 
mind. 


1.  The  team  mind  has  a  limited  capacity  for  attention.  Not  everything 
should  become  conscious.  A  smoothly  functioning  team  can  take  care  of  most 
tasks  at  the  subconscious  (individual)  level,  anticipating  the  requirements  of 
other  team  members.  Neisser  (1976)  described  preattentive  functioning  at  the 
individual  level,  and  this  is  what  we  see  at  the  team  level  as  well — automatic 
performance  of  routine  tasks  by  individuals. 

2.  The  team  mind  depends  on  meta-cognition.  Each  team  must  manage  its  own 
activities.  Anyone,  usually  the  leader,  can  take  steps  to  prevent  memory 
overload  by  recording  critical  information.  Inefficient  use  of  collective 
consciousness  will  occur  in  forms  such  as  micromanagement,  and  an  effective 
team  will  be  on  guard  to  prevent  such  problems.  Teams  are  also  susceptible  to 
interruption,  and  need  to  make  sure  that  this  does  not  lead  to  a  loss  of 
important  information  or  disruption  of  action  sequences. 
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3.  The  team  Bind  is  capable  of  errors.  We  can  distinguish  team  mind  errors 
from  skill/knowledge  errors  of  individuals,  such  as  failure  to  read  a 
checklist  correctly,  or  mishandling  a  landing  under  conditions  of  cross-winds. 
Additionally,  team  Bind  errors  are  related  to  Model  2,  team  decision  errors, 
but  we  have  not  yet  attempted  to  synthesize  the  two  models. 

Team  mind  dysfunctions  are  errors  that  have  to  do  with  interactions 
between  the  three  components  in  Figure  1.  For  example,  in  the  study  of  Boeing 
727  cockpit  crews  (Thordsen  &  Calderwood,  1989),  Crew  #2,  Segment  #3 
encountered  the  problem  of  fuel  leaking  out  of  one  of  the  wing  tanks.  This 
Balfunction  creates  an  imbalanced  weight  condition.  But  the  Captain  wasn't 
sure  whether  it  was  real  failure  or  a  faulty  gauge  for  that  tank.  He  asked 
the  First  Officer,  who  was  doing  the  flying,  "Are  you  having  trouble  flying 
the  plane?"  The  reply  was  "No."  Actually,  we  found  out  later  during  our 
interviews  that  the  First  Officer  was  noticing  an  imbalance,  but  he  was  able 
to  adjust  for  it.  So  he  wasn't  having  trouble.  The  Captain,  who  expected  an 
imbalance,  mistakenly  concluded  there  wasn't  one,  and  decided  they  had  a 
faulty  gauge. 

Consider  another  error  from  Thordsen  and  Calderwood  (1989),  Crew  #3, 
Segment  #2.  The  malfunction  was  a  partial  loss  of  oil  pressure  in  engine  3. 
The  pressure  fell  to  35  psi  which  is  on  the  border  for  shutting  down  the 
engine.  The  Flight  Engineer  wanted  to  shut  off  the  engine,  but  neither  the 
Captain  nor  the  First  Officer  wanted  it  shut  down.  Yet  the  team  shut  it  down 
anyway!  Again,  this  was  a  failure  to  inhibit  an  impulse.  What  happened  was 
that  the  Captain  disagreed  with  the  Flight  Engineer  but  couldn’t  clearly  see 
the  oil  pressure  gauges  which  were  at  the  Flight  Engineer's  station.  So  the 
Captain  backed  down,  and  looked  to  the  First  Officer  for  help.  Tbe  Captain 
said  "I'm  recommending  shutting  down  the  engine,"  hoping  to  get  an  argument. 
The  First  Officer  had  been  flying  the  aircraft  and  not  paying  attention  to  the 
malfunction,  and  took  this  as  an  announcement  rather  than  a  question,  and  said 
nothing.  Since  he  didn't  get  any  reaction,  the  Captain  went  ahead  and  shut 
the  engine  down.  The  action  was  not  necessarily  an  error;  what  was 
significant  was  that  the  two  senior  officers  were  opposed  to  it  but  wound  up 
implementing  it. 

In  Crew  #3,  Segment  02,  when  the  team  shut  down  engine  3,  its  generator 
was  lost  to  them.  And  generator  1  had  been  acting  up  earlier,  and  was  on  a 
standby  status,  not  really  functional.  So  the  airplane  was  landed  with  only 
one  functioning  generator.  In  our  interviews  we  found  that  the  Flight 
Engineer  knew  there  was  only  1  generator,  the  Captain  stated  there  were  one- 
and-a-half  generators  (i.e.,  he  knew  that  a  second  generator  was  on  standby 
but  wasn't  immediately  available),  and  the  First  Officer,  who  was  actually 
flying  the  plane,  believed  it  was  a  2-generator  landing  since  no  one  told  him 
what  the  status  was!  The  team  did  not  make  any  behavioral  error,  but  if  they 
had  it  would  have  arisen  from  faulty  communication  of  situation  assessment. 

A.  The  team  mind  has  intent.  It  is  the  intent  of  the  leader,  usually  the 
official  leader.  In  a  smoothly  running  team,  the  intent  doesn’t  always  have  to 
be  articulated.  The  team  members  will  assume  they  understand  the  intent  until 
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cues  mismatch  their  expectations,  and  then  they  re-check  their  assumptions. 
Example,  Crew  03,  Segment  #3,  the  First  Officer  believed  he  knew  what  the 
Captain  wanted,  and  this  was  confirmed  with  each  other.  This  intent  can  also 
be  unpacked  by  asking  each  member  of  a  team  what  the  leader  was  wanting  to  do 
at  a  given  moment. 

5.  A  smoothly  functioning  team  mind  is  seen  in  positive  efforts  to  improve 
coordination.  It  is  seen  as  anticipating  the  needs  of  others,  synchronizing 
actions,  feeling  free  to  improvise,  offering  effective  feedback  without 
clogging  communications  channels  or  taking  up  too  much  conscious  space. 

6.  The  functioning  of  the  team  mind  becomes  more  effective  and  efficient 
through  training,  as  the  team  learns  to  perform  more  of  the  tasks  at  the 
subconscious  level. 

Methods  for  Studying  the  Team  Mind 

There  are  techniques  for  studying  each  of  the  three  aspects  of  team 
mind:  behavior,  collective  consciousness,  and  subconsciousness. 

The  record  of  the  team's  behavior  is  simply  the  actions  taken.  For  a 
command-and-control  team,  it  is  the  operations  orders  issued  at  various  times 
during  a  battle.  For  an  airplane  it  is  the  control  actions  taken  during  a 
flight.  It  is  helpful  to  be  able  to  link  these  actions  to  environmental 
conditions  so  that  you  can  tell  that  an  engine  was  shut  down,  say,  five 
minutes  after  the  oil  pressure  fell  below  35  psi. 

The  record  of  the  team's  collective  consciousness  is  the  audiotape  or 
videotape  of  what  has  been  articulated.  Since  it  is  tedious  to  listen  to 
tapes,  there  are  advantages  to  charting  the  problem  solving  and  decision 
making  that  the  team  goes  through.  Newell  and  Simon  (1972)  have  worked  out  a 
representation  of  problem  solving  and  decision  making  for  individuals  solving 
cryptarithmetic  problems  while  generating  think-aloud  protocols.  The 
representational  charts  show  each  action  and  the  corresponding  gain  in 
situation  assessment.  We  have  adapted  their  technique  for  groups,  and  have 
charted  planning  sessions  during  command-and-control  exercises  (Thordsen, 
Klein,  Michel,  &  Sullivan,  1988;  Calderwood  &  Thordsen,  1989). 

We  have  recently  taken  the  team  decision  charts  to  a  new  level,  in  our 
recent  research  at  NASA/Ames.  To  illustrate,  we  will  use  the  charts  for  the 
Boeing  727  malfunction  in  which  fuel  was  leaking  from  a  tank  in  one  of  the 
wings.  Three  teams  were  studied.  We  charted  the  way  each  team  developed 
situation  assessment,  and  the  way  each  team  considered  some  key  options.  We 
will  contrast  the  three  teams  with  regard  to  how  they  examined  the  options 
pertaining  to  fuel  flow.  The  first  team,  shown  in  Figure  4,  was  able  to 
consider  the  option  of  cross-feeding  the  fuel  from  the  other  wing,  in  order  to 
minimize  the  imbalance  between  the  weights  of  the  two  wings  (the  recommended 
limit  of  imbalance  is  1,000  lbs.).  This  team  misunderstood  the  standard  and 
thought  they  were  permitted  2,000  lbs.  of  imbalance,  and  were  able  to  achieve 
this  level.  The  second  team,  shown  in  Figure  5,  tried  to  do  the  same  thing 
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HOW  TO  MANAGE  -  Crew  1 


Figure  4  —  Deepening  the  option  of  fuel  transfer  —  Crew 


HOW  TO  MANAGE  -  Crew  2 
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PROTOCOL  ANALYSIS  AND  THE  TEAM  MIND 

Pleura  9  —  Deepening  the  option  of  fuel  transfer  —  Cre«r  2.  HOW  TO  MANAGE  -  CREW  2,  CHART  3 


but  ran  into  a  conflict  between  wanting  to  reduce  imbalance  and  fearing  the 
consequences  of  cross-feeding  fuel  into  an  area  where  there  was  an 
undetermined  leak.  The  team  decision  chart  shows  the  tangled  arguments  and 
counter-arguments  that  made  up  the  collective  consciousness.  This  conflict 
was  actually  between  the  Captain  (source  #1)  and  the  Flight  Engineer  (source 
#3),  with  the  latter  operating  from  a  faulty  understanding  of  the  layout  of 
fuel  tanks  and  cross-feed  lines.  The  Flight  Engineer  prevailed,  and  this  crew 
landed  with  more  than  a  5,000  lb.  imbalance.  The  third  crew,  shown  in  Figure 
6,  was  the  most  inexperienced,  and  their  team  decision  chart  reflects  this — it 
is  the  most  simple.  They  did  not  ever  consider  cross-feeding  fuel  to  reduce 
the  imbalance,  and  while  they  had  the  calmest  cockpit  of  the  three,  they  also 
landed  with  more  than  6,000  lbs.  of  imbalance,  the  least  acceptable  solution. 
We  feel  that  these  types  of  team  decision  charts,  easy  to  produce  and 
indicative  of  the  ongoing  collective  consciousness,  will  be  very  useful  in 
future  research  and  development. 

The  determination  of  the  team's  subconscious  understanding  is  b^st  made 
through  observations  of  videotapes  and  through  interviews  that  enable  us  to 
learn  what  an  individual  knew  that  was  not  communicated  to  the  other  team 
members.  We  have  also  used  formal  strategies,  namely  a  Critical  Decision 
method  (Klein,  Calderwood,  and  MacGregor,  1989)  to  perform  cognitive  probes  of 
individual  decision-making  strategies. 

.1 

The  concept  of  a  team  mind  has  been  introduced  as  a  way  of  viewing  team 
decision-making  performance.  We  hope  that  the  concept  will  be  helpful  for 
introducing  phenomena  from  cognitive  psychology,  e.g.,  limited  attentional 
resources,  the  effects  of  emotions,  workload  variables,  automaticity,  and 
characteristics  of  working  memory.  In  addition,  the  model  of  team  mind  offers 
a  range  of  measurement  approaches  that  hold  promise  for  studying  and 
evaluating  variables  affecting  different  team  performance  variables.  The  team 
mind  is  more  accessible  than  the  individual  mind.  Foushee  and  Helmreich 
(1988)  have  recently  described  the  importance  of  group  process  dynamics  for 
understanding  team  decision  making  and  errors,  and  we  hope  that  the  concept  of 
a  team  mind  can  be  useful  for  research  investigations  into  these  topics. 

WORKSHOP  ON  NATURALISTIC  DECISION  MAKING 

Besides  the  two  studies  completed,  the  two  studies  initiated,  and  the 
two  models  developed,  one  more  major  project  was  accomplished  during  Year  1. 
Klein  Associates  hosted  an  international  conference  on  naturalistic  decision 
making. 

The  Army  Research  Institute  first  suggested  that  it  might  be  able  to 
fund  the  workshop  in  February,  1989,  but  a  contract  was  not  signed  until  late 
July  1989.  Nevertheless,  we  were  able  to  perform  some  impressive  feats  of 
team  decision  making  ourselves,  and  the  workshop  went  off  without  problem  at 
the  end  of  September  1989  despite  the  limited  time  for  planning. 
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Perhaps  the  major  function  of  the  workshop  was  to  bring  together 
researchers  from  disparate  fields  who  have  been  studying  decision  making 
outside  the  laboratory.  In  particular,  a  number  of  related  naturalistic 
models  have  been  put  forward  in  the  last  few  years,  and  the  workshop  allowed 
this  community  to  take  stock  of  what  has  been  happening,  and  to  explore 
opportunities  for  collaboration  and  synthesis.  If  naturalistic  decision 
research  is  indeed  a  third  wave,  following  the  initial  paradigms  of 
mathematical  and  statistical  decision  research,  then  it  is  essential  to 
provide  such  a  forum,  so  that  there  will  be  less  duplicated  effort  and  a 
trading  of  ideas.  The  workshop  served  as  such  a  forum.  Participants  from 
academia,  industry,  and  the  government  attended,  served  on  panels,  met  in 
small  groups,  and  hammered  out  the  major  themes  for  a  book  on  operational 
decision  making.  The  completion  of  this  book  is  scheduled  for  June,  1990. 

The  authors  will  be  Gary  Klein,  Judith  Orasanu,  and  Roberta  Calderwood. 

OBJECTIVES  FOR  YEAR  2 

During  Year  2  we  will  be  completing  three  of  these  projects. 

The  book,  tentatively  entitled  "Decision  Making  in  Complex  Worlds,"  will 
be  finalized  and  submitted  for  publication.  (Two  publishers  have  made  a 
commitment  to  publish  the  book.)  We  also  hope  to  prepare  a  brief  interim 
report,  describing  the  major  themes  of  the  workshop,  especially  the  topics  for 
future  research. 

We  will  also  complete  the  study  of  how  situation  assessment  gets 
communicated,  as  well  as  the  study  of  team  decision  errors  (generating,  we 
hope,  a  more  carefully  worked  out  taxonomy). 

One  "new"  project  is  a  detailed  analysis  of  the  wildland  firefighting 
data.  These  decision  makers  were  the  most  skilled  we  have  yet  encountered, 
and  we  feel  that  it  can  be  very  important  to  document  their  training, 
organizational  dynamics,  and  capabilities. 

A  second  "new"  research  thrust  for  Year  2  is  an  examination  of 
individual  decision  errors,  reviewing  data  from  the  studies  we  have  already 
performed  under  various  sponsorship.  We  are  particularly  eager  to  document 
ways  that  a  recognitional  decision  strategy  can  lead  to  ineffective 
performance. 

The  third  activity  planned  for  Year  2  is  to  study  mental  simulation  in 
much  the  way  we  started  our  investigations  of  naturalistic  decision  making — 
observations  of  proficient  decision  makers  working  out  a  conceptual  image  of 
what  would  happen  if  an  option  was  implemented. 

Two  other  projects  are  being  examined  for  feasibility.  One  is  the 
potential  for  using  the  ARI  database  on  command-and-control  exercises  held  at 
the  National  Training  Center.  We  would  like  to  examine  Commander’s  Intent 
through  these  data,  and  we  also  want  to  see  if  we  can  document  the  proportion 
of  intent  and  goals  that  are  actually  implemented  during  the  course  of  a 
simulated  battle. 
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The  second  project  is  the  possibility  of  studying  situation  assessment 
training  within  the  context  of  an  Intensive  Care  Unit  at  a  hospital  in  the 
Dayton  area.  Since  situation  assessment  training  is  on  the  agenda  for  Year  3, 
we  may  want  to  get  a  head  start  and  perform  some  observational  research. 

SUMMARY 

This  has  been  an  extremely  fast  start  for  the  contract.  We  initiated 
four  studies,  rather  than  the  three  we  had  planned.  We  developed  two  new 
models  of  team  decision  making.  And  we  conducted  a  workshop  on  naturalistic 
decision  making  for  the  leading  researchers  in  this  area.  We  will  not  keep  up 
this  pace  during  Year  2,  but  we  are  feeling  very  excited  about  the 
accomplishments  of  Year  1,  and  about  their  implications  for  helping  us  to 
understand  operational  decision  making. 

We  would  also  like  to  mention  some  direct  applications  of  our  research 
for  ARI.  We  have  consulted  with  the  National  Fire  Academy,  where  instructors 
are  beginning  to  use  the  RPD  model  for  course  presentations  on  what  to  expect 
during  crisis  decision  making.  Also,  we  have  had  several  opportunities  to 
present  a  training  module  on  crisis  decision  making  at  the  Central  Training 
Academy,  in  Albuquerque,  New  Mexico.  The  Central  Training  Academy  serves  the 
Department  of  Energy  by  providing  managerial  training  to  administrators  of 
nuclear  production  facilities,  to  help  prepare  them  to  deal  with  accidents, 
and' emergencies  such  as  terrorist  attacks. 
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